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BACKGROUND 

1. Field of the Invention 

This invention relates generally to the field of image analysis methods and more 
particularly to methods of creating a mask surrounding an area of interest in an image, such that 
5 when the mask is applied to the image (e.g., though a summation operation) only the area of 
interest remains. The invention is applicable to digital images generally. One area where it is of 
particular utility is analysis of images of biological specimens. For example, the mask can serve 
to isolate and highlight areas of interest, such as cancerous cells, in a magnified image of a 
cellular or tissue specimen. 

10 

2. Description of Related Art 

In the biology fields, including cytology, histology, and pathology, digital images of 
tissue and cellular specimens are typically obtained from a microscope equipped with a color 
camera which records red, green, and blue planes for these images. Frequently, the objects in the 

15 specimen can fall into two general types: normal cells and abnormal cells. The abnormal cells 
may for example be cells which have indicia of cancer, due to their size, shape and/or color. 

It is also common practice to apply one or more stains to the specimen on the slide so that 
the objects of interest have a contrasting color from background objects or objects of less interest 
so that they are more readily identified and observed. For example, normal cells are often 

20 stained (or, counterstained as is usually said) with a stain such Hematoxylin and appear light 
blue, while abnormal cells (i.e. positive cells) are stained with a different stain, such as 3-amino 
9-ethylcarbazol (AEC) so that the abnormal cells have a different color, e.g. reddish brown. 
Other color combinations are possible and known in the art. 
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A quantitative analysis of a limited region of interest in an image is performed in some 
image processing procedures. For example, image analysis techniques may be applied to a 
limited region of interest in order to quantify the amount of DNA or a protein present in a region 
of interest. The algorithms used in such techniques are easier to execute if the unwanted areas in 
5 the image, i.e., those area extraneous to the area of interest, are eliminated. Hence, masking 
techniques are used to isolate the region of interest. In known masking techniques, the region of 
interest in the image is undisturbed, but image information in the peripheral areas outside the 
perimeter of the region of interest is deleted, e.g., converted to black. To achieve this, a mask is 
created that is basically black outside of the region of interest and white (pixel values of 255) in 

10 the region of interest. The mask is applied to the image using a logical product operation, e.g., 
AND, wherein 255 operated by AND with any pixel value yields the pixel value and 0 operated 
by AND with any pixel value yields 0. 

The prior art has had difficulty in easily, reliably and automatically creating a mask for a 
region of interest that is bounded by closed curve. The hard part is telling what "inside" is, i.e., 

15 defining the portion of the image comprising the region of interest that is interior of the closed 
curve. This is particularly so if the closed curve is essentially any arbitrary closed curve. Given 
that the region of interest in biological specimens can take virtually any conceivable shape, the 
algorithms have to be able to create a mask for a region of interest that is bounded by any valid 
closed curve. An arbitrary closed curve is reducible to a complex polygon in a digital image in 

20 which the pixels are arranged in rows and columns. Solution of the problem of whether a 
particular appoint in the image is inside or within any arbitrary closed curve (complex polygon) 
is a non-trivial problem. There are heuristics approaches that work some but not all the time, 
but that is not satisfactory if the image analysis procedures are to work reliably and 
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automatically. Other solutions to the problem are more reliable than heuristics methods, but 
they are exceedingly complex to implement. 

The present invention provides a method of creating a mask for a portion of interest in an 
image that is very easy to understand conceptually and code in software, very rapid to execute in 
5 a computer, and works reliably all the time. As such, it presents a useful contribution to the art. 
The present invention takes advantage of the insight and discovery of a way to define the area 
outside of the area of interest and create a mask based on the territory peripheral to the closed 
curve, rather than attempting to find the interior of the region of interest specified by the closed 
curve. 
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SUMMARY 

In a first aspect, a method is provided for creating a mask isolating a region of interest in 
an image. The mask serves to delete information in the image that is peripheral to the region of 
interest. The method includes the step of obtaining a digital image containing the region of 
5 interest. For example, this first image may be a magnified color image of a cellular specimen, 
however the invention is applicable generally. The region of interest is represented by pixels 
having pixel values. This image is typically displayed on a computer display and a first closed 
curve is defined in the image, forming a perimeter around the region of interest. The step of 
defining the closed curve or perimeter will typically be performed by a user on the display of a 

10 workstation (e.g., using a mouse to outline the closed curve). If there are any gaps in the curve, 
the gaps are closed through appropriate algorithms so as to form a single closed curve. 

The method includes a step of creating a second image comprising a plurality of pixels. 
This second image, occasionally referred to herein as an underlay image, is used to create the 
mask. The second image has a boundary (e.g., a rectangular boundary) corresponding to at least 

1 5 a portion of the first or original image. The first closed curve around the region of interest in the 
first image is represented in the second image as a matching second closed curve. The second 
closed curve in the underlay image is contained within the boundary of the underlay image. In 
one possible embodiment, a software tool is used such that when the user draws the perimeter of 
the region of interest in the first or original image, the matching closed curve is created 

20 simultaneously in the underlay image. 

As noted above, the present invention makes use of the insight that it is easier to 
recognize the areas outside of the closed curve than it is to identify the areas inside of the closed 
curve. The invention carries this out by assigning all pixels on the boundary of the underlay 
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image with a first pixel value, e.g., the minimum pixel value, e.g., 0 in an 8 bit quantization 
scheme (black) and similarly assigning the first (minimum) pixel values to all the other pixels 
peripheral to the second closed curve, e.g., by flood filling techniques. Additionally, all pixels 
in the underlay image within the region bounded by the second closed curve are assigned a 
5 second pixel value, e.g., 255 (white). This can be achieved by initializing the underlay image 
such that all pixels in the underlay image have pixel values of 255, creating the closed curve in 
the underlay image as a black line, and flood filling the peripheral pixels with minimum values 
(black). 

The underlay image with the pixel values assigned as recited is then saved in memory as 
10 a mask. Application of the mask to the first image (e.g., by a logical AND operation) deletes 
image information for areas peripheral to the region of interest and leaves pixel values for the 
region of interest undisturbed. The areas peripheral to the region of interest appear black, 
whereas the region of interest pixels appear as in the original. Alternatively, after the mask has 
been applied to the original image, the resulting image can be reversed such that the peripheral 
15 areas appear white and the region of interest appears as in a negative. 

In a related aspect, a workstation for creating a mask for a digital image composed of 
pixels having pixel values is provided. The mask isolates a region of interest in the image and 
deletes information in the image peripheral to the region of interest. The workstation comprises a 
processing unit, a user interface including a display and a pointing device associated with the 
20 display, and a memory storing the image. The workstation further includes machine readable 
instructions for execution by the processing unit. The machine-readable instructions comprising 
instructions for: 

1) displaying the image on the display; 
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2) providing the user the ability to define with the pointing device and the 
display a first closed curve forming a perimeter around the region of interest; 

3) creating a second (underlay) image comprising a plurality of pixels, the second 
image having a boundary corresponding to at least a portion of the image encompassing and 

5 enclosing the region of interest, 

4) creating in the second image a second closed curve within the boundary 
matching the first closed curve; 

5) assigning all pixels in the second image on the boundary and peripheral to the 
second closed curve with a first pixel value (e.g., 0) and assigning all pixels in the second image 

10 within the region bounded by the second closed curve a second pixel value, e.g., 255 (white) and 

6) applying the mask to the image and thereby deleting information peripheral to 
the region of interest and leaving pixel values for the region of interest undisturbed. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 is diagram on biological image environment in which the invention can be 
employed. 

5 Figure 2 is a magnified image of a tissue sample that may contain one or more regions of 

interest shown displayed on the user interface of the general-purpose computer of Figure 1 . 

Figure 3 is a further magnified portion of the image of Figure 2, shown displayed on the 
user interface of the general -purpose computer, showing the user tracing a closed curve forming 
the perimeter of a region of interest with a mouse. 
10 Figure 4 is an illustration of a second or underlay image initialized to all pixel values 

equal to a maximum value, showing the outline of a closed curve in black that matches the 
closed curve drawn by the user in Figure 3. 

Figure 5 is an illustration of the underlay image of Figure 4 which forms a mask after a 
flood-fill operation has been performed to assign pixel values for the perimeter pixels and all 
15 pixels outside of the closed curve a minimum value (e.g., 0 in an 8 bit gray scale system). The 
area within the closed curve is white (pixels values equal to 255 in an 8 bit system). 

Figure 6 shows the result of application of the mask of Figure 5 to the image of Figure 3 
in the region surrounding the area of interest. The area peripheral to the region of interest are 
black (contain no image information), whereas the pixels in the region of interest are 
20 undisturbed. 
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DETAILED DESCRIPTION OF 
PREFERRED EMBODIMENTS 



5 While the present image mask invention is applicable generally to digital images 

containing a region of interest, a presently preferred embodiment is in the field of imaging of 
biological samples and the invention will therefore be described in this context. Figure 1 is an 
illustration of digital image acquisition apparatus that includes a microscope 10, X-Y stage 12 
for holding a slide 14 containing a biological specimen, and a plurality of objective lenses for 

10 focusing a magnified image of the specimen on the slide 14 onto a charge-coupled device color 
imaging camera (not shown). The apparatus of Figure 1 is conventional and known in the art. 
An image acquired by the camera in the microscope 10 is captured and stored in digital form and 
therefore can be sent over a communications link to any general-purpose computer 20. The 
computer 20 includes a central processing unit (CPU 22), a hard disk memory (not shown) for 

15 storing the image captured by the microscope, and a user interface including as screen display 
24, computer keyboard 26 and mouse 28. The computer 20 is conventional and may be any off- 
the-shelf general-purpose computer. The memory of the computer 20 includes standard image 
processing and display software, which may consists of a package or suite of such software, for 
displaying images on the display 24 of the computer 20. Such software packages are known in 

20 the art and commercially available or may be standard equipment with the computer 20. In 
Figure 1, the display 24 includes a display of the magnified image 30 of the specimen on the 
slide. 

Figure 2 is another view of the screen display 24, showing the magnified image 30 of a 
tissue sample on the slide 24. The image 30 may contain one or more regions of interest to a 
25 user (e.g., pathologist or cytotechnician) operating the computer. The entire slide does not fit 
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into the window shown in Figure 24, therefore the image display software includes tools 32 and 
34 allowing the user to scroll up or down or from side to side so as to be able to view the entire 
region of the slide. 

Figure 3 is a further magnified portion of the image 30 of Figure 2. The user may access 
5 this more magnified view of the slide by activating a suitable icon or entering commands via the 
keyboard. The user has now identified a region of interest 40 in the slide. The present invention 
provides a method for providing a mask around the region of interest 40 such that areas 
peripheral to the region 40 are black (contain no image information) but the region of interest 
40 remains undisturbed. Rather than masking the entire slide except for the region of interest, a 

1 0 preferred embodiment masks only a rectangular area surrounding the region of interest 40. 

The user marks the perimeter of the region of interest 40 by using the mouse pointing 
device 38, for example clicking the mouse while moving the cursor 38 around the periphery of 
the region of interest. This action identifies particular pixel addresses that create a closed curve 
36 defining the region of interest 40. In the event that the user accidentally created gaps in the 

15 curve 36, the gaps are closed by constructing lines connecting the end points of defining the gap. 
This could be done by a suitable algorithm in the software or by prompting the user to click 
again on the image and close any gaps. 

The present invention creates the mask by using a second or underlay image 50, shown in 
Figure 4. The underlay image in the illustrated embodiment is a rectangular image, comprised 

20 of individual pixels 52 (shown greatly enlarged in Figure 4), that corresponds to at least a 
portion of the original image 30. In one possible embodiment, the underlay image 50 could have 
the same number of pixels and be the same size as the original image. In other embodiments, 
for example where multiple areas of interest may be present in the original image, the underlay 
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image 50 only corresponds to a portion of the original image that immediately surrounds the 
region of interest designated by the user as in Figure 3. In the embodiment of Figure 4, as the 
user draws the closed curve in Figure 3, the same closed curve 54 is created in the underlay 
image 50. The underlay image size may be dynamically adjusted. The boundary 62 of the 
5 underlay image 50 may be dynamically varied substantially simultaneous with the defining of 
the closed curve 36, to thereby insure that the underlay image completely encompasses the 
closed curve 36 (curve 54 in the underlay image). This is the embodiment shown in Figure 4. 
This closed curve 54 is separated from the boundary 62 of the underlay image, as indicated by 
the gaps 56 between the closed curve 54 and the boundary 62. The gaps can be created by 

10 adding an arbitrary number of rows and columns to the row and column pixel coordinates 
defining the upper, lower, left hand and right hand bounds of the closed curve. The closed curve 
54 separates the interior 60 of the closed curve from the periphery 58 of the closed curve. 

The underlay image of Figure 4 is not necessarily displayed to the user and can simply 
exist as a file in the memory of the computer. When the underlay image is initially created, the 

15 pixel values for all pixels in the underlay image are initialized to a maximum value (white). 
When the closed curve 36 is drawn as shown in Figure 3, the pixels coordinates assigned by the 
user with their mouse are carried over to the pixel coordinates in the underlay image and each of 
these pixels are assigned a minimum pixel value (0) corresponding to black. 

With the underlay image created as shown in Figure 4, the mask is created by flood 

20 filling the pixel values for all the pixels in the peripheral region 58 to black (0). The algorithm 
automatically knows the pixel coordinates for the boundary of the underlay image, and by 
definition the boundary is exterior of the closed curve 54 since the boundary was created by 
adding some positive integer number of pixels to the row and column pixel coordinates of the 
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upper, lower, left hand and right hand extremes of the closed curve. Consequently a simple 
flood fill algorithm changes all pixel values for the boundary pixels and all pixels exterior of the 
pixels defining the closed curve 54. The result is the underlay image forming a mask shown in 
Figure 5, showing interior region 60, and black pixels in the peripheral region 58. 
5 The mask is applied to the original image (or a portion thereof) in a logical product 

operation, wherein 255 ANDed with any pixel value x returns x, and 0 ANDed with any pixel 
value x returns 0. The result is shown in Figure 6. All pixels exterior of the closed curve are 
black (maximum pixel value) and contain no image information. The interior points containing 
the region of interest 46 are undisturbed. 

10 Thus, from the above description I have described a method for creating a mask (Figure 

5) isolating a region of interest in an image, as shown in Figure 6. The mask serves to delete 
information in the image that is peripheral to the region of interest, as shown in Figure 6. The 
method includes the step of obtaining a digital image containing the region of interest. For 
example, this first image may be a magnified color image of a cellular specimen, acquired using 

15 the apparatus of Figure 1, however the invention is applicable generally. The region of interest 
40 (Figure 3) is represented by pixels in the original image. This image is typically displayed on 
a computer display. A first closed curve 36 is defined in the image, forming a perimeter around 
the region of interest. The step of defining the closed curve or perimeter will typically be 
performed by a user on the display of a workstation (e.g., using a mouse to outline the closed 

20 curve), as explained above. If there are any gaps in the curve, the gaps are closed through 
appropriate algorithms so as to form a single closed curve. 

The method includes a step of creating a second image (Figure 4) comprising a plurality 
of pixels. This second or underlay image, is used to create the mask. The second image has a 
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boundary 62 (e.g., a rectangular boundary) corresponding to at least a portion of the first or 
original image. The first closed curve 40 around the region of interest in the first image is 
represented in the second image as a matching second closed curve 54. The second closed curve 
in the second or underlay image is contained within the boundary of the second image. In one 
5 possible embodiment, a software tool is used such that when the user draws the perimeter of the 
region of interest in the first or original image, the matching closed curve 54 is created 
simultaneously in the second or underlay image of Figure 4. 

As noted above, the present invention makes use of the insight that it is easier to 
recognize the areas outside of the closed curve than it is to identify the areas inside of the closed 

10 curve. The invention carries this out by assigning all pixels on the boundary of the second 
image with a first pixel value, e.g., minimum pixel value, e.g., 0 in an 8 bit quantization scheme 
(black) and similarly assigning the same pixel value to all the other pixels peripheral to the 
second closed curve, e.g., by flood filling techniques. Any point in the first row (y = 0), the first 
column (x = 0), the last row (y = imageheight - 1), or the last column (x = image width - 1) can 

1 5 be chosen as a starting point for the flood fill algorithm. Once chosen, a floodfill is performed of 
all pixels of that color (in this case black) to convert them to black (the color of the perimeter 
line 54). 

Additionally, all pixels in the second image within the region 60 bounded by the second 
closed curve are assigned a second pixel value, e.g., 255 (white) . This can be achieved by 
20 initializing the second image such that all pixels in the second image have pixel values of 255, 
creating the closed curve in the underlay image as a black line, and flood filling the peripheral 
pixels with minimum values (black). 

The second image with the pixel values assigned as recited is shown in Figure 5 and then 
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can saved in memory or a buffer as a mask. 

Application of the mask of Fig. 5 to the first image (e.g., by a logical AND operation ) 
deletes image information for areas peripheral to the region of interest and leaves pixel values for 
the region of interest undisturbed. See Figure 6. The areas 58 peripheral to the region of interest 
5 appear black, whereas the region of interest 40 pixels appear as in the original. Alternatively, 
after the mask has been applied to the original image, the resulting image can be reversed such 
that the peripheral areas appear white and the region of interest appears as in a negative. 

It will also be appreciated that I have described a workstation or computer 20 for creating 
a mask for a digital image. The workstation comprises a processing unit 22, a user interface 
10 including a display 24 and a pointing device 28/38 associated with the display 24, and a memory 
storing the image. The workstation further includes machine readable instructions for execution 
by the processing unit. The machine-readable instructions comprising instructions for: 

1) displaying the image on the display, as shown in Figures 2 and 3; 

2) providing the user the ability to define with the pointing device and the 
15 display a first closed curve 36 forming a perimeter around the region of interest, as shown in 

Figure 3; 

3) creating a second (underlay) image (Figure 4) comprising a plurality of pixels 
52, the second image having a boundary 62 corresponding to at least a portion of the image, 

4) creating in the second image a second closed curve 54 within the boundary 
20 matching the first closed curve (as shown in Figure 4); 

5) assigning all pixels in the second image on the boundary and peripheral to the 
second closed curve with a first pixel value corresponding and assigning all pixels in the second 
image within the region bounded by said second closed curve a second pixel value, as shown in 
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Figure 5, and 

6) applying the mask to the image as shown in Figure 6, e.g., through a logical 
AND operation , thereby deleting information peripheral to the region of interest and leaving 
pixel values for the region of interest undisturbed. 

5 While presently preferred embodiments have been described with particularity, variation 

from the details of the preferred embodiment are contemplated without departure from the true 
scope and spirit of the invention. For example, the nature of the original image, and the means 
by which the image is acquired and stored is unimportant. The method works with any 
arbitrary user-defined closed curve or polygon. Additionally, other pixel values could be chosen, 

10 or the black and white pixel values for the masks could be reversed and a different logical or 
summation operation used to apply the mask to the original image to thereby delete the 
information exterior to the region of interest. 
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